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The findings from this review do not refiect the fuii body of research evidence on Tools of the Mind. 


What is this study about? 

The study authors examined whether Tools of the Mind 
increased academic achievement, cognitive flexibility, 
working memory, attention control, and cognitive pro- 
cessing speed for kindergarten students. This inter- 
vention is a pedagogical approach that emphasizes 
teacher-led interactions between classmates. The 
program is based on the idea that students’ “executive 
function,” or self-regulation, is the main mechanism by 
which they can increase learning. 

The study authors randomly assigned 29 schools in 
12 school districts to either implement the Tools of 
the Mind program during the kindergarten year for 2 
consecutive years or to a comparison group that did 
not implement a similar program.^ Up to six students 
per class were assessed on the outcome measures 
at the beginning of the kindergarten year (baseline), 
at the end of the kindergarten year, and at the begin- 
ning of first grade. When students participating in 
the study did not complete follow-up assessments, 
the study included other students who had been 
wait-listed or were additionally recruited at follow- 
up. With these students, a total of 759 kindergarten- 
ers were included in the analyses. 

The authors assessed the impacts of Tools of the 
Mind by comparing the intervention and comparison 
group on measures of academic achievement in 
mathematics, alphabetics, comprehension, cognitive 
flexibility, working memory, attention control, and 
cognitive processing speed. 


Features of Tools of the Mind 


Tools of the Mind is a curriculum that aims to 
promote cognitive and academic outcomes for 
kindergarten students by focusing on “executive 
function” as the centrai process to the deveiopment 
of students’ academic and social competencies. 
Tools of the Mind is based on the ideas of 
Russian psychologist Lev Vygotsky. Instruction 
is individualized through teacher scaffolding, 
and dramatic play is a main component of the 
curricuium. Teachers meet with students weekiy 
to deveiop individuaiized iearning plans and to 
review and critique work. Teachers then provide 
instruction using interactions among classmates. 
The curriculum includes math, literacy, and science 
activities that align with Common Core standards. 
Professional development for teachers and 
paraprofessionals are provided by Tools of the Mind 
staff during the first 2 years of implementation. 
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What did the study find? 



None of the analyses presented in this study meet 
WWC group design standards and therefore, the 
study findings are not presented in this WWC report. 


The research described in this 
report does not meet WWC 
group design standards 


Some students in the analytic sampie joined the 
study as a second cohort in the year after random 
assignment had occurred. As a result, the analytic 
sample included non-randomly assigned joiners 
in both the intervention and comparison groups. 
Therefore, baseline equivalence is required for the 
study to meet WWC group design standards with 
reservations. Baseline equivalence could not be 
demonstrated for the analytic sample because 
baseline data are not available for joiners. Methods 
of accounting for missing data on baseline outcome 
measures can only be used for randomized 
controlled trials with low attrition.'^ For this reason, 
the study is rated does not meet WWC group design 
standards and therefore, the findings from this study 
are not presented in this WWC report. 
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Endnotes 

■' Blair, C., & Raver, C. C. (2014). Closing the achievement gap through modification of neurocognitive and neuroendocrine function: 
Results from a cluster randomized controlled trial of an innovative approach to the education of children in kindergarten. PloS ONE, 
9(11), e112393. doi:10.1371/journal.pone.01 12393 

^ Single study reviews examine evidence published in a study (supplemented, if necessary, by information obtained directly from the 
authors) to assess whether the study design meets WWC design standards. The review reports the WWC’s assessment of whether 
the study meets WWC design standards and summarizes the study findings following WWC conventions for reporting evidence on 
effectiveness. This study was reviewed using the single study review protocol (version 2.0). A quick review of this study was released 
in February 2015, and this report is the follow-up review that replaces that initial assessment. This review includes additional informa- 
tion on the study design that was obtained from the authors after the quick review was released. 

® In two districts, all kindergarten classrooms were in a single school, so the team randomly assigned classrooms instead of schools 
and treated the single schools as separate classrooms in the analysis. 

^ The study used a statistical technique referred to as Full-Information Maximum Likelihood to include students with missing baseline 
data in the analysis. Baseline equivalence cannot be demonstrated on an analytic sample that includes missing data. No analyses 
were reported that were based only on complete cases. 

Recommended Citation 

U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2015, November). 

WWC review of the report: Closing the achievement gap through modification of neurocognitive and neuroen- 
docrine function: Results from a cluster randomized controlled trial of an innovative approach to the education 
of children in kindergarten. Retrieved from http://whatworks.ed.gov 
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Glossary of Terms 

Attrition 


Clustering adjustment 
Confounding factor 

Design 
Domain 
Effect size 

Eligibility 

Equivalence 

Improvement index 


Multiple comparison 
adjustment 

Quasi-experimental 
design (QED) 

Randomized controlled 
trial (RCT) 

Single-case design 
(SCD) 

Standard deviation 


Statistical significance 
Substantively important 


Attrition occurs when an outcome variable is not avaiiabie for aii participants initiaiiy assigned 
to the intervention and comparison groups. The WWC considers the total attrition rate and 
the difference in attrition rates across groups within a study. 

If intervention assignment is made at a cluster level and the analysis is conducted at the student 
level, the WWC will adjust the statistical significance to account for this mismatch, if necessary. 

A confounding factor is a component of a study that is completely aligned with one of the 
study conditions, making it impossible to separate how much of the observed effect was 
due to the intervention and how much was due to the factor. 

The design of a study is the method by which intervention and comparison groups were assigned. 
A domain is a group of closely related outcomes. 

The effect size is a measure of the magnitude of an effect. The WWC uses a standardized 
measure to facilitate comparisons across studies and outcomes. 

A study is eligible for review if it falls within the scope of the review protocol and uses either 
an experimental or matched comparison group design. 

A demonstration that the analytic sample groups are similar on observed characteristics 
defined in the review area protocol. 

Along a percentile distribution of individuals, the improvement index represents the gain 
or loss of the average individual due to the intervention. As the average individual starts at 
the 50th percentile, the measure ranges from -50 to +50. 

When a study includes multiple outcomes or comparison groups, the WWC will adjust 
the statistical significance to account for the multiple comparisons, if necessary. 

A quasi-experimental design (QED) is a research design in which study participants are 
assigned to intervention and comparison groups through a process that is not random. 

A randomized controlled trial (RCT) is an experiment in which eligible study participants are 
randomly assigned to intervention and comparison groups. 

A research approach in which an outcome variable is measured repeatedly within and 
across different conditions that are defined by the presence or absence of an intervention. 

The standard deviation of a measure shows how much variation exists across observations 
in the sample. A low standard deviation indicates that the observations in the sample tend 
to be very close to the mean; a high standard deviation indicates that the observations in 
the sample are spread out over a large range of values. 

Statistical significance is the probability that the difference between groups is a result of 
chance rather than a real difference between the groups. The WWC labels a finding statistically 
significant if the likelihood that the difference is due to chance is less than 5% (p < .05). 

A substantively important finding is one that has an effect size of 0.25 or greater, regardless 
of statistical significance. 


Please see the WWC Procedures and Standards Handbook (version 3.0) for additional details. 


November 2015 


Page 4 


WWC Single Study Review 




Intervention 

Report 


Practice 

Guide 


Quick Single Study 
Review Review 



A single study review of an individual study includes the WWC’s assessment of the quality of the research design 
and technical details about the study’s design and findings. 

This single study review was prepared for the WWC by Mathematica Policy Research under contract ED-IES-13-C-0010. 
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